Conceptual metaphor, human-computer interaction and music : applying conceptual metaphor to the design and analysis of music interactions

Interaction design for domains that involve complex abstractions can present significant challenges. This problem is particularly acute in domains where users lack effective means to conceptualise and articulate relevant abstractions. In this thesis, we investigate the use of domain-specific conceptual metaphors to address the challenge of presenting complex abstractions, using tonal harmony as an extended case study. This thesis presents a methodology for applying domain-specific conceptual metaphors to interactions designs for music. This domain involves complex abstractions where users with any degree of domain knowledge may have difficulty in articulating concepts. The methodology comprises several parts. Firstly, the thesis explores methods for systematically guiding conversation between musicians to elicit speech that describes music using conceptual metaphors. Recommendations for the most suitable methods are made. Secondly, the thesis presents a methodology for identifying image schemas and conceptual metaphors from transcriptions of conversations between musicians. The methodology covers rules for identifying source image schemas and extrapolating conceptual metaphors. Thirdly, the thesis presents a methodology for evaluating existing music interaction designs using domain-specific conceptual metaphors. We demonstrate that this approach can be used to identify potential areas for improvement as well as tensions in the design between certain tasks or abstractions. Fourthly, the thesis presents a case study for the development of a conceptual metaphor-influenced design process. In the case study, a set of materials are developed to be used by participants in the design process to facilitate the mapping of conceptual metaphors to elements of an interaction design without requiring knowledge of Conceptual Metaphor Theory. Conceptual metaphor, human-computer interaction and music Finally, a pilot study is presented integrating the results of the conceptual metaphorinfluenced design process into a consistent and useful prototype system. Compromises and refinements to the design proposals made during the design process are discussed and the resulting system design is detailed. Conceptual metaphor, human-computer interaction and music Declarations Aspects of this work have been previously published in the following: WILKIE, K., HOLLAND, S. and MULHOLLAND, P., 2013. Towards a participatory approach for interaction design based on conceptual metaphor theory: a case study from music interaction. In: S. HOLLAND, K. WILKIE, P. MULHOLLAND and A. SEAGO, eds, Music and Human-Computer Interaction. London: Springer, pp. 259–270. WILKIE, K., HOLLAND, S. and MULHOLLAND, P., 2011. We can work it out: towards a participatory approach to designing music interactions. Proceedings of the Workshop on “Music Interaction Design” at the British Computer Society HCI Conference 2011, Newcastle Upon Tyne. WILKIE, K., HOLLAND, S. and MULHOLLAND, P., 2010. What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal, 34(4), pp. 34-48. WILKIE, K., HOLLAND, S. and MULHOLLAND, P., 2009. Analysis of conceptual metaphors to improve music software: the role of prior experience in inclusive music interaction. Proceedings of the Workshop on “Prior Experience and Inclusive Design” at the British Computer Society HCI Conference 2009 Workshop, Cambridge. WILKIE, K., HOLLAND, S. and MULHOLLAND, P., 2009. Evaluating musical software using conceptual metaphors. Proceedings of the British Computer Society HCI Conference 2009, Cambridge. British Computer Society, pp. 232–237. Elements of the present work appeared previously in the above as follows: Figure 7:1 page 73, previously published in Wilkie et al. (2010). © The MIT Press 2010 Table 5:1 page 42, adapted from a version previously published in Wilkie et al. (2010). © The MIT Press 2010 Table 7:1 page 71, adapted from a version previously published in Wilkie et al. (2010). © The MIT Press 2010 Conceptual metaphor, human-computer interaction and music Acknowledgements There are many people who provided support and assistance throughout the course of this work. Firstly, I would like to express my very grateful thanks to my supervisors, Simon Holland and Paul Mulholland. Without their invaluable support and advice, this work would not have been possible. I would like to thank Jörn Hurtienne for his feedback during the early stages of this research, and for his advice on image schema and conceptual metaphor identification in particular. I would like to thank Marian Petre and my fellow colleagues from the online forum for the informative and entertaining conversations on various aspects of research and for their encouragement during the more challenging phases of this research. I would like to thank all the members of research, administration and support staff that have provided advice and assistance during the course of this research. I would like to thank all the participants that took part in the various studies that formed part of this research. I would like to thank my parents, my brother and his family, J and P, O and my friends for their support. I would also like to apologise for not visiting them as often as I should have and for the numerous late birthday cards. I would also like to thank those who were not able to see me complete this journey, B and J, T and E, C, J. I would like to thank my good friend Eileen for proof reading elements of this document. Finally, I would like to thank Jim. Without his continued support, encouragement, proof reading skills and uncanny ability to provide a cup of tea at just the right time, this work would not have been possible. “We are starstuff. We are the universe made manifest, trying to figure itself out. And as we have both learned, sometimes the universe requires a change of perspective.” Straczynski, J. M. and Fontana D. C. (1994) Babylon 5. Episode. A Distant Star. Conceptual metaphor, human-computer interaction and music